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ABSTRACT 



The results of a student evaluation of a television 
course on tests and measurements offered by the Extension Service, 
California State College, Los Angeles, California, are presented. The 
evaluation form, which was general in nature, was completed by 157 
students who finished the course- Questions asked were: (1) How did 

you find out about the course? (2) What is your occupation? (3) How 
useful was the course in general? (4) How helpful w,as the textbook? 

(5) How helpful were the unit tests? (6) What did you like best about 
the course? (7) How could the course have been improved? (8) How 
difficult was the course? (9) Would you enroll for another course of 
this design in the field of education and psychology? (10) What other 
television courses do you wish to be offered? Tables summarizing the 
student responses and a discussion of the data are presented in the 
document. Results demonstrated that the course was well received and 
perceived as a useful learning experience. Problems in future 
presentation of the course are also discussed. (AE) 
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STUDENT EVALUATION OF THE TELEVISION COURSE 
"USING TESTS INTELLIGENTLY" 

The television course, "Using Tests Intelligently", vras recorded 
by the Broadcast Ser’/ice Center, California State College, Los Angeles, 
in the latter part of 1969i and the tapes are now available for use. 

The course was initially broadcast over KABC-TV (Los Angeles) for UO 
half-hour lessons at 6:30 AM beginning February 16, 1970. There were 
five lessons a week and the course continued for eight weeks. The 
course was offered under the auspices of the Extension Service, 

California State College, Los Angeles, and provided four quarter units 
of upper division credit under the college catalog designation, 

Education 409 • Educational Tests and Measurements . 

The TV lessons are relaxed and informal but carefully structured* 
Each lesson begins with a listing of ob,jectives stated in behavioral 
terms. At the conclusion of each lesson,, students answer a series 
of short objective review questions presented on the TV screen followed 
by a listing of correct responses. Supplementary information is supplied 
to students through assignments in a textbook especially prepared for 
use in the course. 

In the initial presentation of the course, students were 
required to take four unit tests on Saturdays at conveniently located 
test centers. As soon a-s the students handed in their answer sheets 
at the test centers, they were givon answer keys providing not only a 
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listing of correct responses, but also code designations to indicate 
textbook readings covering each item. Students were permitted to 
keep the test booklets for further study. In this manner, students 
were given immediate feedback of test results with an opportunity 
for studying areas of weakness. At the conclusion of the course, 
students were administered a final summary test and also at that 
time filled out an unsigned student evaluation form vrtiich has pro- 
vided the basis for this evaluation. 

An innovative feature of the course was the use of a computer- 
based item storage and retrieval system vdiich helped make it convenient 
to develop numerous forms of each test so that students experiencing 
difficulty on particular tests could be given opportunities to repeat 
parallel forrris of the same tests. With this system, most conscientious 
students were able to pass all tests. The sequence followed was 
test - study - retest, etc. until a passing level of achievement 
was reached. Most students, of course, were able to pass the test 
on the first try. However, the knowledge that they woiiLd be retested 
if necessary helped relieve anxiety and, the instructor believes, led 
to more effective test performance by students. 

Altogether 172 students enrolled for the initial presentation 
of the course, but eight of these officially dropped. Another seven 
were given unofficial withdrawals because they missed several tests 
and indicated nc wish to make these up. Thus, altogether 157 students 
or 91 percent of those originally enrolled, finished the course. 
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Distribution of grades for the course lyas as follows: 

A — 59 (38 percent) 

B ~ 87 (55 percent) 

C “ 1 i (7 perc ent ) 

D — 0 

F — 0 

The evaluation form provided students with an opportunity 
to respond to the following questions: 

1. How did you find out about the course? 

2. VJhat is your occupation? 

3 . How useful was the course in general? 

4^ How helpful was the textbook? 

5 . How helpful were the unit tests? 

6 . What did you like most about the course? 

7 . How could the course have been most improved? 

8 . How difficult was the course? 

9* Would you enroll for another course of this design in 
the field of education and psychology? 

10. What other television courses do you wish to be offered? 

The next sections will provide tables summarizing the student 
answers to each question. After each table there will be a brief 







discussion of thrs data. 
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1. How did students find out about the course? (146 
students responding) 

51% A, Given brochure at school 
\2% B. TV Announcement 
10^ C. Informed by friend 
9^ D, Informed by school official 
6^ E. Picked up brochure in public library 
k% F. Read about it in College Extension Service Bulletin 
2% G. Other (Newspaper articles, informed by course 
instructor, etc r ) 

A systematic and intensive effort was made to inform prospective 
students of the course* Initially, the course was announced in the 
regular College Extension Service Bulletin, which was sent to a 
majority of the schools in Los Angeles County and bordering areas 
together with several descriptive brochures for each school. There- 
after, the TV instructor made a personal telephone call to each 
school district, described the course to the director of curriculum 
and arranged for widespread distribution of the brochures to individual 
teachers. Additionally, all the public libraries in Los Angeles 
County were supplied with brochures. In this manner, approximately 
25,000 brochures were distributed. 

Several newspaper articles describing the course were published, 
including a feature article in th«i South Bay Daily Breeze, a newspaper 
in the South Los Angeles area with approximately 60,000 subscribers. 

The television station KABC-TV was helpful in publicizing the course. 



• 5 - 



For the week preceding the course, four op five brief spot announce- 
ments were gi'f en daily and the announcements continued through the 
first 10 days of the course. 

Despite these efforts to publicize the course, the tital en- 
rollment was not large. The most effective means of communication 
with prospective students apparently was through distribution of the 
descriptive brochures to the school. This distribution was materially 
assisted by school personnel whose help was solicited through telephone 
calls by the television instructor. 

As part of the evaluation fora, students were asked to write 
their ideas, if any, as to why more students did not enroll in the 
class. Relatively few responded to this question. Several mentioned 
that some prospective students were afraid to take the course because 
they feared it might be too mathematical. Others mentioned that many 
persons were probably reluctant to give up the four Saturday mornings 
to take the Unix, tests required in the course. Although the cost 
($57-85) was mentioned by only a very few as a reason for not taking 
the course, the instructor suspects that this was an important factor 
in limiting enrollment. 

It should be added that the instructor is aware of a number 
of persons vriio viewed the course regularly even though they were not 
officially enrolled. In one school district, a group of teachers 
met several times to diLscuss the content of particular lessons. No 
doubt there were others of whom the instructor is not personally 
aware who became interested in the course and viewed it regularly. 




2. What were the student's occupations? (134 responding) 

72^ A. Teachers 

17^ B. Preparing to become teachers 

b% C. Housewives 

5^ D, Others (Social Worker, Registered Nurse, Employment 

Counselor, Students) 

It is evident that the course was of prime attraction to 
teachers and those planning to become teachers. 

3- How useful was the course in general? (146 responding) 





A. 


Extremely useful 




B. 


Very useful 


19% 


C. 


Some'^at useful 


'\tsf 

iO 


D. 


Not very useful 


0% 


E. 


A waste of time 



A large majority of the students found the course very or 
ey':,remelj useful. A number of students wrote enthusiastic letters 
in support of the course, made encouraging phone calls, and engaged 
in personal conversations with the TV instructor indicating strong 
approval. Of the 146 students, only three rated the course as "not 
very useful" and none rated j.t a "waste of time" . 

4. How helpful was the textbook? (146 responding) 

46^ A. Extremely helpful 
46^ B. Very helpful 
8^ C. Somewhat helpful 
0% D. Not very helpful 
0^ E. 



Useless 
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Many favorable comments ab^^ut the textbook were made to the 
instructor: The statement of objectives at the beginning of each 

chapter, the careful organization of content, the simple style and 
the use of two sets of practice questions in each chapter coded to 
specific paragraphs were specifically mentioned as adding to the 
value of the textbook as a learning aid. 

5* How helpfiiL were the luiit tests? 



5 ^ 


A, 


Extremely helpful 


33 % 


B. 


Very helpfiJ. 




G. 


Somewhat helpful 


1 % 


D, 


Not very helpful 


0 % 


E, 


Useless 



Although a few students at the begirtning of the course com- 
plained about the requirements of taking the four unit tests on 
Saturdays, it appears that this was one of the most popular features 
of the course- The unit tests provided immediate feedback to students 
regarding their progress, and indicated needed areas for review- The 
fact that students were permitted to repeat tests on which they did 
poorly probably helped relieve the anxiety often associated with test 
taking. Apparently, as a result of the course, tests began to be 
perceived as learning aids - not threats. A number of teachers stated 
that the testing procedures of the course changed their whole outlook 
on appropriate uses of tests, and they were finding in their own class- 
rocmis that using tests as learning aids not only improved class morale, 
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but also seemed to lead to better student achievement. The instructor 
believes that the most significant feature of th® course is that it 
provides a model of how, in fact, tests really can be used with primary 
emphasis on helping students learn. 

6. V/hat did the students like most about the course? (117 responding) 

The following list represents an attempt to summarize the 
general sense of the desired features of the course according to 
student opinion. Several of the students listed more than one aspect 
as being most liked in the course^ The numbers in the list do not 
represent percent, but the actual number of those responding. 

34 Convenience 

31 B. Testing with immediate feedback. 

^ C. TV instructor’s method of presentation (infoimial 
atmosphere, visual aids, etc.) 

21 D. The careful structure of each lesson beginning with a 

listing of objectives which is repeated at the conclusion 
of the lesson. 

14 E. The attempts to lessen anxiety about tests. 

12 F,. The originality of the ideas about testing. 

4 G-. The fact that the instructor practices what he preaches 

in the course. 

4: H. Instructor’s availability to answer questions over the telephone. 

3 I* Feeling of being individually tutored. 

2 J. Repetition of difficult points. 

I K. Opportunity to work independently. 

1_ L. The textbook 

1^ M. The short length of each lesson. 






On the basis of these comments^ it appears that a course can 
be carefully planned, follow a fixed structure and still be perceived 
by the student as an informal and pleasant experience. Students 
seemed to feel that they were being treated with appropriate personal 
consideration, and probably more so than in the typical classroom lecture 
situation. It seems clear also that the course was presented so that 
tests were perceived as learning aids and not threats. 

7^ How could the course have been improved? (77 responding) 

The following list attempts to summarize ideas for improving 
the course. Several students listed more than one response as to 
how they would like to see the course improved. The numbers do not 
represent percents, but the actual number of those making each suggestion. 



O 



- 10 - 



30 A* Present course materials more slowly, particularly 

concepts relating to quantitative aspects of measurement. 

10 B. Have more opportunity for meeting with instructor for 
discussion and questions. 

9 C. Have course at a later hour. 

7 D. Provide more examples on applied use of measurement 
in the classroom. 

6 E. Have unit tests printed so as to be more legible. 

5 F. Present course in a more dynamic manner. 

3 0. Provide supplementary materials explaining more difficult 

points. 

3 H. Have less emphasis on the instructor’s personal opinions. 

2 I* Get textbooks mailed out to the students sooner. 

1 J. Make testing and grading less threatening. 

JL K. Give the final exam in a room which is quieter and 
where there are fewer distractions. 

1 L. Require the instructor to be less optimistic about 
everyone passing. 

1 M. Give fewer tests. 

1 N. Have students take all unit tests ly mail. 

0. Avoid so much use of the word ’’necessarily’* both in 
the television presentation and in the textbook. 

As may be seen from the list, quite a few of the students 
suggested a slower pace for the course. The instructor, in planning 
the course, intended it to be ’’meaty” . It was anticipated that students 
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vould find it necessary to study and review some aspects of the course 
independently* This seems appropriate for a senior-graduate level 
course* However, the instructor does believe it highly desirable that 
one of the lessons in particular should be redone because of the overly 
rapid pace of presentation* 

The suggestion that students have more opportunity to meet with 
the instructor for discussion and questions is one that is often made 
for television courses* Although the instructor highly favors pro- 
viding students with this kind of opportunity, his experience in 
other TV courses suggests that such discussions are not well attended 
on a voluntary basis either because of problems of scheduling discussions 
at a convenient time, or because of lack of student intere^^t. 

The suggestion that the course be given at a later hour is 
probably not a very good one because for a great majority of the 
students 6:30 AM is likely the most convenient time. However, if 
it were possible, showing the same lesson at a later hour in the day 
would be useful, and particularly so for the students who might want 
to see the same lesson twice. 

The instructor believes that the suggestion to provide more 
examples in the applied use of measureh^ent in the curriculum is a 
good one, and would like to prepare four or five additional lessons 
in the course for this purpose. 

An obvious problem in the course was that some of the unit tests 
were printed so light as to be difficult to read* This will be easy 
to correct in future course presentations. 
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The general suggestion that the course be presented in a more 
dynamic manner summarizes several individual suggestions such as: 
more interesting visual aids should be used, the instructor should 
tell more jokes, and so forth. 

The rest of the suggested improvements have been made either 
by very few individuals or single individuals and, for the most part, 
are probably not significant. 

8. How difficult was the course? (146 responding) In this 
case, responses are given as percents. 

k% A. Very difficult 
37^ B* Difficult 
^h% C • Of average difficulty 
3^ D. Easy 
0% E. Very easy 

These ratings are consistent with the instructor's perception of 
what might be the expected level of difficulty of a course of this 
type for state college students at the senior - graduate level. 

9. Would students enroll for another course of this type? 



52% 


A. 


Definitely yes 


3k% 


B. 


Probably yes 


10% 


C. 


Undecided 


3% 


D. 


Probably not 


1% 


E. 


Definitely not 



Although it is obviously uncertain vriiether or not students would 
actually enroll for another course if offered, the response to this 
question seems to indicate a generally favorable attitude toward 
television instruction of the type presented. 
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10# What other television courses would students like to be 
offered? (70 responding) The following is a list of courses under 
general categories suggested by the students. Several students listed 
more than one course, Tl;e numbers on the list represent actual numbers 
responding rather than percent. 

Education (General) 

11 A, Any education course ^ich could be used toward meeting 
certification requirements. 

6 B, Diagnosis and remediation of learning difficulties. 

^ C. Teaching the culturally deprived, 

2 0, Curriculum planning. 

2 S, Handling student behavior problems, 

2 F, Psychological or sociological foundations of education. 



Education (Specific subject fields) 

12 A, Teaching reading. 

7 B, Mathematics 
7 C, History 
^ D, Literature 

2 E, English as a second language. 
2 F, Science 
2 G, Art 



O 



Psychology, Counseling, and Guidance (A variety of courses) 



12 


A. 


Psychology 


10 


B, 


Counseling and Guidance 


6 


C. 


Advanced courses in testing. 
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The fact that most of the courses listed here are in education or 
related fields is consistent vdth the fact that most of those taking 
the course were teachers or planning to become teachers. It would 
be pointed out that only 70 responded to this question, and it is 
the instructor's impression that the responses were generally '^off 
the cuff'' and did not represent careful considerations of possible 
desired courses. 

|\ iture Presentation of the Course 

Res\alts of the questionnaire demonstrate that the course was 
well received and perceived as a useful learning experience. Tho 
course provided a practical demonstration how use of well defined 
goals, carefully organized presentations with as much student involve 
ment as possible, and appropriate feedback combine to provide an 
effective instructional procedure. Many of the students reported 
that the course provided a model presentation for modifying their own 
teaching methods to advantage. 

As is well known, there is currently considerable interest 
in education in improving use of tests as learning aids and as a 
means for encouraging teacher accountability for student success. 
Considering the current significance of course content, the general 
success of the course, and the amount of effort and other resources 
devoted to its development, the instructor highly recommends further 
use of the course. 

The instructor is well aware that although many will agree 
with the desirability of additional presentations of the course, it 
will probably be difficult to secure approval and adequate financial 



14 



- 15 - 



support for further uses. Althouf^h the cost of using the video tape 
is not especially high, and may be negotiated with the Broadcast 
Service Centex; California State College, Los Angeles, current systems 
of budget allocations in various educational institutions often do 
not provide for convenient methods of support. Furthermore, achieving 
approval by faculty for course presentation may often be a very time 
consuming process. However, if in fact, use of the course constitutes 
a significant educational advance, then it does appear desirable to 
find means for overcoming the obvious '^roadblocks’^ with reasonable 
dispatch and moving ahead to additional presentations of the course. 



Quentin C. Stodola 

California State Collage at Dominguez Hills 
June 1970 
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